Method for generating prediction result for predicting occurrence of fatal symptoms of subject in advance and device using same

ABSTRACT

The present invention relates to a method for generating a prediction result for predicting an occurrence of fatal symptoms of a subject in advance, a method for performing data classification by using data augmentation in mechanical learning for the same, and a computing device using the same. Particularly, the computing device according to the present invention acquires vital signs of the subject, converts the same into individuated data, generates analysis information from the individuated data on the basis of a machine learning model, generates a prediction result by referring to the analysis information, and provides the prediction result to an external entity.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Continuation Application of U.S. patentapplication Ser. No. 16/638,250, filed on Feb. 11, 2020, now pending,which is a National Stage Application of International Application No.PCT/KR2018/008918, filed on Aug. 7, 2018, which claims priority ofKorean Patent Application Nos. 10-2017-0102265, filed on Aug. 11, 2017,and 10-2017-0106529, filed on Aug. 23, 2017. The contents of which areall incorporated by references herein in their entireties.

BACKGROUND Technical Field

Example embodiments relate to a method of generating a prediction resultfor early predicting an occurrence of fatal symptoms of a subject, amethod of classifying data using data augmentation for machine learning,and a computing apparatus using the methods.

Related Art

In medical clinical trials, an occurrence of fatal symptoms of asubject, for example, a patient, such as cardiac arrest and sepsis, maysignificantly decrease survival discharge. For example, cardiac arrestaccounts for 20 to 30% of survival discharge rates. Since such cardiacarrest has a change in vital signs before its occurrence, earlyprediction thereof is possible. However, a prediction method accordingto the related art mainly depends on experience or knowledge of amedical specialist, for example, a nurse or a doctor in charge.Therefore, the risk of the subject or the patient may be evaluated quietdifferently depending on an individual's capability. Also, there is ashortage of emergency medical staff for such an evaluation itself.

Briefly describing the prediction method according to the related art,the risk of fatal symptoms, such as cardiac arrest, is determined byassigning scores according to a conventional rule-based vital signvalue. Here, there is a high rate of false negative or false positive.Here, the false negative refers to a case of not predicting the risk ofa patient of which actual fatal symptoms are to occur and the falsepositive refers to a case of predicting an occurrence of actual fatalsymptoms even though the fatal symptoms do not occur.

In reality, an imbalance of data becomes an issue in classifying a classof data through a machine learning.

For example, a case of predicting cardiac arrest of a patientcorresponds to a 2-class classification in which a series of vital signsacquired from the patient are classified into vital signs (first class)corresponding to cardiac arrest or classified into vital signs (secondclass) of a normal patient not having cardiac arrest, through a machinelearning using the series of vital signs as learning data. However,since many subjects are subjects not showing corresponding symptoms,vital sign data corresponding to cardiac arrest are few. That is, asevere class imbalance of learning data occurs.

That is, a machine learning algorithm performs learning with a set ofdata biased to one class and thus, a resulting classification model maydecrease an overall accuracy and may further decrease an accuracy ofclassifying data corresponding to the other class. In a classification,it is important to match a class (a minority class, the first class inthe above example) to which a minority of data belongs as well as aclass (a majority class, the second class in the above example) to whicha majority of data belongs. Therefore, the aforementioned issue needs tobe outperformed.

Accordingly, proposed herein is a fatal symptoms early prediction resultgenerating method that may early predict fatal symptoms furtheraccurately compared to a conventional method. Also, to this end, thereis proposed a method of improving a conventional GAN as a method ofincreasing an accuracy of a classification model through a machinelearning by generating data similar to true data, that is, by performingan effective data augmentation as a method of overcoming a severe classimbalance of data.

-   (Reference document) Non-patent document 1: Goodfellow, Ian J.;    Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David;    Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua (2014). “Generative    Adversarial Networks”

SUMMARY Technical Subject

Example embodiments are to detect a patient with imminent fatalsymptoms, such as cardiac arrest, and to increase a survival rate of thepatient by reducing existing high false negatives.

Also, example embodiments are to increase an accuracy of aclassification model through existing machine learning using dataaugmentation of generating a minority class of learning data similar totrue data although a class imbalance of learning data is high in machinelearning.

Also, example embodiments are to improve medical treatment conditions bysaving unnecessary consultation hours through a decrease in existinghigh false positives.

Technical Solution

Characteristic constitutions of this disclosure to accomplish theaforementioned objectives and to achieve characteristic effects of thedisclosure are as follows:

According to an aspect of example embodiments, there is provided amethod of generating a prediction result for early predicting anoccurrence of fatal symptoms of a subject, the method including: (a)acquiring, by a computing apparatus, or supporting another apparatusinteracting with the computing apparatus to acquire vital signs of thesubject; (b) converting, by the computing apparatus, or supporting theother apparatus to convert the acquired vital signs to individuationdata that is data individuated for the subject; (c) generating, by thecomputing apparatus, or supporting the other apparatus to generateanalysis information about an early prediction of the fatal symptomsfrom the individuation data based on a machine learning model for theearly prediction of the fatal symptoms, and generating or supporting theother apparatus to generate the prediction result as a result ofpredicting the occurrence of the fatal symptoms during a duration from apoint in time t corresponding to a single specific vital sign among thevital signs to a point in time t+n that is a point in time after adesired time interval n by referring to the generated analysisinformation; and (d) providing, by the computing apparatus, orsupporting the other apparatus to provide the generated predictionresult to an external entity.

Desirably, the method may further include (e) updating, by the computingapparatus, or supporting the other apparatus to update the machinelearning model based on evaluation information about the predictionresult.

According to another aspect of example embodiments, there is provided acomputing apparatus for generating a prediction result for earlypredicting an occurrence of fatal symptoms of a subject, the computingapparatus including: a communicator configured to acquire vital signs ofthe subject; and a processor configured to convert or support anotherapparatus interacting through the communicator to convert the acquiredvital signs to individuation data that is data individuated for thesubject. The processor is configured to generate or support the otherapparatus to generate analysis information about an early prediction ofthe fatal symptoms from the individuation data based on a machinelearning model for the early prediction of the fatal symptoms, generateor support the other apparatus to generate the prediction result as aresult of predicting the occurrence of the fatal symptoms during aduration from a point in time t corresponding to a single specific vitalsign among the vital signs to a point in time t+n that is a point intime after a desired time interval n by referring to the generatedanalysis information, and provide the generated prediction result to anexternal entity.

Desirably, the processor may be configured to update or support theother apparatus to update the machine learning model based on evaluationinformation about the prediction result.

According to another aspect of example embodiments, there is provided amethod of classifying data using data augmentation for a machinelearning, the method including: (a) acquiring, by a computing apparatus,or supporting another apparatus interacting with the computing apparatusto acquire true data; (b) training, by the computing apparatus, orallowing the other apparatus to train a generator and a discriminator ofa modified generative adversarial network (GAN) based on information ofa label corresponding to the acquired true data and the true data,wherein, in the modified GAN, the generator includes a sub-generatorconfigured to generate similar data corresponding to each of a pluralityof labels, the sub-generator is configured to generate similar databelonging to a label corresponding to the sub-generator, and thediscriminator is configured to predict a specific label that is a labelcorresponding to data to be discriminated by the discriminator among theplurality of labels; (c) training, by the computing apparatus, orallowing the other apparatus to train a machine learning model bygenerating the similar data using the trained modified GAN and by using(i) the true data and the similar data or (ii) the similar data aslearning data of a predetermined machine learning model forclassification; and (d), in response to acquiring data to be classified,generating, by the computing apparatus, or supporting the otherapparatus to generate classification information of the data to beclassified by classifying the data to be classified based on the trainedmachine learning model. For example, the machine learning model forclassification may use data generated by the generator of the modifiedGAN trained based on true data as learning data.

According to another aspect of example embodiments, there is provided acomputing apparatus for classifying data using data augmentation for amachine learning, the computing apparatus including: a communicatorconfigured to acquire true data; and a processor configured to train orallow another apparatus interacting through the communicator to train agenerator and a discriminator of a modified generative adversarialnetwork (GAN) based on information of a label corresponding to theacquired true data and the true data, wherein, in the modified GAN, thegenerator includes a sub-generator configured to generate similar datacorresponding to each of a plurality of labels, the sub-generator isconfigured to generate similar data belonging to a label correspondingto the sub-generator, and the discriminator is configured to predict aspecific label that is a label corresponding to data to be discriminatedby the discriminator among the plurality of labels. The processor isconfigured to train or allow the other apparatus to train a machinelearning model by generating the similar data using the trained modifiedGAN and by using (i) the true data and the similar data or (ii) thesimilar data as learning data of a predetermined machine learning modelfor classification, and, in response to acquiring data to be classified,generate or support the other apparatus to generate classificationinformation of the data to be classified by classifying the data to beclassified based on the trained machine learning model.

According to another aspect of example embodiments, there is provided acomputer program stored in media including instructions that cause acomputing apparatus to perform the method.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will be described in more in detail with referenceto the following figures that are simply a portion of the exampleembodiments and those of ordinary skill in the art (hereinafter, “thoseskilled in the art”) to which this disclosure pertains may readilyacquire other figures based on the figures without an inventive workbeing made:

FIG. 1 illustrates an example of describing a recurrent neural network(RNN) that is a machine learning model according to an exampleembodiment.

FIG. 2 is a diagram illustrating an example of a computing apparatusconfigured to perform a method of generating a prediction result forearly predicting an occurrence of fatal symptoms of a subject(hereinafter, also referred to as a “fatal symptoms early predictionresult generating method”) and a method of classifying data using a dataaugmentation (hereinafter, also referred to as a “data classificationmethod”) according to an example embodiment.

FIG. 3 is a diagram illustrating an example of hardware and softwarearchitectures of a computing apparatus configured to perform a fatalsymptoms early prediction result generating method according to anexample embodiment.

FIG. 4 is a flowchart illustrating an example of a fatal symptoms earlyprediction result generating method according to an example embodiment.

FIG. 5 illustrates an example of a method of performing a dataaugmentation using a modified generative adversarial network (GAN)according to an example embodiment.

DETAILED DESCRIPTION

The following detailed description of this disclosure is described withreference to the accompanying drawings in which specific exampleembodiments of the disclosure are illustrated as examples, to fullydescribe purposes, technical solutions, and advantages of thedisclosure. The example embodiments are described in detail enough forthose skilled in the art to carry out the disclosure.

The term “fatal symptoms” when used in the detailed description andclaims and modifications thereof are not limited to cardiac arrest thatis one example of targets to which the disclosure applies and should beunderstood as a concept including any kind of clinical phenomena thatmay cause a great risk in a subject's life due to time series changes,such as sepsis, etc.

Also, the term “vital signs” when used in the detailed description andclaims should not be interpreted to be limited to a general meaning of ameasurement value, such as a body temperature, an electrocardiogram(ECG), a respiration, a pulse rate, a blood pressure, an oxygensaturation, a skin conductivity, and the like, of a subject and shouldbe understood to include electroencephalography (EEG) signals, an amountof specific substance among biological samples acquirable through othermeasurements, a concentration, and the like.

Here, “biological samples” should be understood as various kinds ofsubstances that may be collected from the subject, for example, blood,serum, urine, lymph, cerebrospinal fluid, saliva, semen, vaginal fluid,etc., of the subject.

The term “training/learning” when used in the detailed description andclaims refers to performing a machine learning through computingaccording to a procedure and can be understood by those skilled in theart that the term is not intended to refer to mental acts, such as humaneducational activities.

Also, the terms “comprises/includes” when used in the detaileddescription and claims and modifications thereof are not intended toexclude other technical features, additions, components, or operations.Those skilled in the art may clearly understand a portion of otherpurposes, advantages, and features of the disclosure from thisspecification and another portion thereof from implementations of thedisclosure. The following examples and drawings are provided as examplesonly and not to limit the disclosure.

Further, the disclosure may include any possible combinations of exampleembodiments described herein. It should be understood that, althoughvarious example embodiments differ from each other, they do not need tobe exclusive. For example, a specific shape, structure, and featuredescribed herein may be implemented as another example embodimentwithout departing from the spirit and scope of the disclosure. Also, itshould be understood that a position or an arrangement of an individualcomponent of each disclosed example embodiment may be modified withoutdeparting from the spirit and scope of the disclosure. Accordingly, thefollowing detailed description is not to be construed as being limitingand the scope of the disclosure, if properly described, is limited bythe claims, their equivalents, and all variations within the scope ofthe claims. In the drawings, like reference numerals refer to likeelements throughout.

Unless the context clearly indicates otherwise, the singular forms “a”,“an”, and “the”, are intended to include the plural forms as well. Also,when description related to a known configuration or function is deemedto render the present disclosure ambiguous, the correspondingdescription is omitted.

Hereinafter, example embodiments of the disclosure are described indetail with reference to the accompanying drawings such that thoseskilled in the art may easily perform the example embodiments.

FIG. 1 illustrates an example of describing a recurrent neural network(RNN) that is a machine learning model according to an exampleembodiment.

Referring to FIG. 1 , in a machine learning model used herein, a deepneural network model may be briefly described to be in a form in whichartificial neural networks are stacked in multiple layers. That is, adeep structured neural network may be represented as a deep neuralnetwork or a DNN that is a network in a deep structure, and, referringto FIG. 1 , may be trained using a method of automatically learningfeatures of vital signs and a relationships between the vital signsthrough learning of a large amount of data in a structure that includesa multilayered network and, through this, reducing an error of anobjective function, that is, a prediction accuracy of fatal symptoms. Itis also expressed as a concatenation between nerve cells of the humanbrain and the deep neural network (DNN) is becoming a next generationmodel of artificial intelligence (AI).

Among DNN models used herein, a recurrent neural network (RNN) may beused to analyze sequentially input data as shown in FIG. 1 . The RNNmodel is in a structure to detect a feature of data according to a timesequence and to selectively apply a main feature to be referred to whenanalyzing a current point in time among features of previous points intimes. For example, referring to FIG. 1 , when analyzing data input at apoint in time t+1, the data may be analyzed by learning main featuresanalyzed at points in times t−1 and t. As described above, according toexample embodiments, it is possible to extract a change in vital signsover time using a structure of the RNN and to use the extracted changefor prediction of fatal symptoms.

For example, the RNN unfolded according to time-series order, time flow,or time axis may be understood as a DNN having an infinite number oflayers. In FIG. 1 , x_(t) denotes an input vector at the point in time tand s_(t) denotes a hidden state (i.e., memory of a neural network) atthe point in time t.

Describing again, the RNN of FIG. 1 follows s_(t)=f(Ux_(t)+Ws_(t-1)) andy=g(Vs_(t)). For reference, y is indicated with o in FIG. 1 . Here, fdenotes an activation function (e.g., tanh( ) and ReLU function) and U,V, and W denote parameters of a neural network. Here, U, V, and W referto parameters that are shared equally across stages of all of the pointsin times in the RNN, which differs from those in a feedforward neuralnetwork. Also, g denotes an activation function (typically, a softmaxfunction) for an output layer, and y denotes an output vector of theneural network at the point in time t. A method using the RNN accordingto an example embodiment is further described below.

FIG. 2 is a diagram illustrating an example of a computing apparatusconfigured to perform a fatal symptoms early prediction resultgenerating method according to an example embodiment.

Referring to FIG. 2 , a computing apparatus 200 according to an exampleembodiment includes a communicator 210 and a processor 220, and maydirectly or indirectly communicate with an external computing apparatus(not shown) through the communicator 210.

In detail, the computing apparatus 200 may achieve a desired systemperformance using a combination of typical computer hardware (e.g., anapparatus including a computer processor, a memory, a storage, an inputdevice and an output device, components of other existing computingapparatuses, etc.; an electronic communication apparatus such as arouter, a switch, etc.; an electronic information storage system such asa network attachment storage (NAS) and a storage area network (SAN)) andcomputer software (i.e., instructions that enable a computing apparatusto function in a specific manner).

The communicator 210 of the computing apparatus 200 may transmit andreceive a request and a response with another interacting computingapparatus. As an example, the request and the response may beimplemented using the same transmission control protocol (TCP) session.However, it is provided as an example only. For example, the request andthe response may be transmitted and received as a user datagram protocol(UDP) datagram. In addition, in a broad sense, the communicator 210 mayinclude a keyboard, a mouse, and other external input devices to receivea command or an instruction, etc.

Also, the processor 220 of the computing apparatus 200 may include ahardware configuration, such as a data bus, a micro processing unit(MPU) or a central processing unit (CPU), a cache memory, and the like.Also, the processor 220 may further include a software configuration ofan application that performs a specific object, an operating system(OS), and the like.

FIG. 3 is a diagram illustrating an example of hardware and softwarearchitectures of a computing apparatus configured to perform a fatalsymptoms early prediction result generating method according to anexample embodiment.

Describing a method and a configuration of an apparatus according to anexample embodiment with reference to FIG. 3 , the computing apparatus200 may include a vital sign acquisition module 310 as a component.Those skilled in the art may understand that the vital sign acquisitionmodule 310 may be implemented through an interaction with thecommunicator 210 included in the computing apparatus 200, or aninteraction between the communicator 210 and the processor 220.

The vital sign acquisition module 310 may acquire vital signs of asubject. For example, vital signs may be acquired from an electronicmedical record (EMR) of the subject. However, it is provided as anexample only.

The acquired vital signs may be forwarded to an individuation module320. The individuation module 320 converts the vital signs toindividuation data that is data individuated or personalized for thesubject. For example, the vital signs may be acquired from the subjectthat is a human (Homo sapiens). However, those skilled in the art mayunderstand that they are not limited thereto. That is, “individuation”may be performed with respect to a specific animal subject incorrespondence to performing “personalization” with respect to aspecific human subject.

Such conversion is performed since a normal vital sign differs from avital sign just before an occurrence of fatal symptoms for each subject.For example, one subject may breathe 45 times per minute when normal,while another subject may breathe 45 times just before cardiac arrest.As described above, the acquired vital signs need to be modified to fitan individual subject instead of being simply used.

As an example of the conversion to individuate vital signs for thesubject if necessary, the individuation module 320 may convert the vitalsigns to the individuation data by calculating a deviation of vitalsigns of an entire time duration by subtracting an average of vitalsigns of an initial desired time duration among the vital signs from thevital signs of the entire time duration, and by calculating a standardscore (z-score) of the vital signs of the entire time duration of thesubject as the individuation data by referring to an average and avariance of vital signs of a plurality of other subjects.

Further describing in detail, with the assumption that vital signs offirst 10 hours of a subject are normal, a difference from an average,that is, a deviation may be calculated using a value acquired bysubtracting an average of the vital signs of the 10 hours from vitalsigns of remaining hours. A z-score for the entire vital signs of thesubject may be acquired by calculating an average and a variance ofvital signs with respect to all of the subjects. In this manner, asubsequent vital sign may be considered as a value relatively deviatedfrom normal vital signs of each subject rather than using an absolutevalue.

When the individuation data that is data individuated for each subjectis input to a first analysis module 330 a and a second analysis module330 b, analysis information about the early prediction may be generated.A prediction module 340 may generate a prediction result about anoccurrence probability of fatal symptoms by a specific point in timebased on the analysis information. A process of generating the analysisinformation and the prediction result is further described below.

An update module and learning module 350 may function to pretrain amachine learning model to be used in response to an occurrence of fatalsymptoms of the subject by performing the method according to theexample embodiments or to update the machine learning model based onevaluation information about the prediction result according toperforming of the method.

Hereinafter, a method of generating a result of early predicting fatalsymptoms according to an example embodiment is described with referenceto FIG. 4 . FIG. 4 is a flowchart illustrating an example of a fatalsymptoms early prediction result generating method according to anexample embodiment.

Referring to FIG. 4 , a prediction result generating method according toan example embodiment includes operation S410 of acquiring, by the vitalsign acquisition module 310 implemented by the communicator 210 of thecomputing apparatus 200, vital signs of the subject. For example, thevital signs may be time series vital signs from a point in time tO to apoint in time t. However, without being limited thereto, the vital signsmay be vital signs at a single point in time. Those skilled in the artmay understand from this specification that a prediction result aboutfatal symptoms may be generated even with respect to vital signs at asingle point in time.

Referring to FIG. 4 , the prediction result generating method furtherincludes operation S420 of converting, by the individuation module 320implemented by the processor 220 of the computing apparatus 200, orsupporting another apparatus interacting through the communicator 210 toconvert the acquired vital signs to individuation data that is dataindividuated for the subject.

According to an example embodiment, as described above, in operationS420, the individuation module 320 may calculate a deviation of vitalsigns of an entire time duration by subtracting an average of vitalsigns of an initial desired time duration among the vital signs from thevital signs of the entire time duration, and may calculate a standardscore (z-score) of the vital signs of the entire time duration of thesubject as the individuation data by referring to an average and avariance of vital signs of a plurality of other subjects. Here, thoseskilled in the art may understand that a method of generatingindividuation data by converting to fit a characteristic of a patientfor each subject is not limited to the aforementioned method.

Referring again to FIG. 4 , the fatal symptoms early prediction resultgenerating method further includes operation S432 of generating orsupporting the other apparatus to generate analysis information about anearly prediction of the fatal symptoms from the individuation data basedon a machine learning model for the early prediction of the fatalsymptoms and operation S434 of generating or supporting the otherapparatus to generate the prediction result as a result of predictingthe occurrence of the fatal symptoms during a duration from a point intime t corresponding to a single specific vital sign among the vitalsigns to a point in time t+n that is a point in time after a desiredtime interval n by referring to the generated analysis information.

According to an example embodiment, a recurrent neural network (RNN)model may be included as an analysis model in the machine learningmodel. The analysis modules 330 a and 330 b that implement the analysismodel may be executed by the processor 220. As described above, the RNNmodel follows s_(t)=f(Ux_(t)+Ws_(t-1)) and y=g(Vs_(t)). Here, the x_(t)denotes the individuation data that is an input vector at the point intime t or a value processed from the individuation data. Here, the valueprocessed from the individuation data may refer to, for example, avariance (from a previous point in time to a current point in time) ofthe individuation data or an amount of change in the variance.

Also, the s_(t) denotes a hidden state corresponding to a memory of theRNN model at the point in time t, the s_(t-1) denotes the hidden stateat the point in time t−1, the U, V, and W denote neural networkparameters shared equally across all the points in times of the RNNmodel, the f denotes a predetermined first activation function that isselected to calculate the hidden state, the y denotes an output layerthat is a latent feature according to the RNN model at the point in timet as the analysis information, and the g denotes a predetermined secondactivation function that is selected to calculate the output layer.

In detail, the first analysis module 330 a between the analysis modules330 a and 330 b functions to apply a relationship between theindividuation data by referring to individuation data at the point intime t, and may correspond to, for example, Ux_(t) in the RNN model.Also, the second analysis module 330 b between the analysis modules 330a and 330 b functions to apply a change in the individuation data overtime by referring to individuation data up to the point in time t−1 andmay correspond to, for example, Ws_(t-1) in the RNN model.

Here, the first activation function f may be a tanh( ) or ReLU functiongenerally used. Also, the second activation function g may be a softmaxfunction generally used. The first activation function and the secondactivation function may be selectively applied based on each purpose andcomplexity of calculation.

Also, according to the example embodiment, the machine learning modelmay further include a second neural network model including at least onefully connected layer for calculating an occurrence probability of thefatal symptoms from the output layer (y) as at least a portion of aprediction model. The prediction module 340 that implements theprediction model may be executed by the processor 220.

Prior to performing the fatal symptoms early prediction resultgenerating method according to example embodiments, operation S405 ofpretraining the machine learning model may be required. To this end, anupdating and learning module 350, or the learning module may be executedby the processor 220.

For learning of the RNN model, the updating and learning module 350 maytrain the RNN model through backpropagation through time (BPTT) of usingindividual individuation data for a plurality of existing subjects aslearning data. Through this, the U, V and W may be determined.

Also, for learning of the second neural network model, the updating andlearning module 350 may train the second neural network model throughbackpropagation of using individual individuation data for a pluralityof existing subjects and an occurrence/non-occurrence of fatal symptomsat each point in time of the existing subjects as learning data.

Further describing operations S432 and S434 according to the exampleembodiment, the processor 220 may acquire or support the other apparatusto acquire the output layer at the point in time t+n as the analysisinformation by using the individuation data or the value processed fromthe individuation data as an input of the analysis modules 330 a and 330b in operation S432. In operation S434, the processor 220 may generateor support the other apparatus to generate an occurrence probability ofthe fatal symptoms up to the point in time t+n as the prediction resultby using the output layer at the point in time t+n as an input of theprediction module 340.

Referring again to FIG. 4 , the prediction result generating methodfurther includes operation S440 of providing, by the processor 220, thegenerated prediction result to an external entity. Here, the externalentity may include a user of the computing apparatus 200, anadministrator, a medical specialist in charge of the subject, and thelike. In addition thereto, any entity capable of acquiring theprediction result should be understood to be included.

Meanwhile, similar to the example embodiment of FIG. 4 , when theoccurrence probability of the fatal symptoms is predicted to be higherthan a predetermined probability, the prediction result may be providedto the external entity.

The fatal symptoms early prediction result generating method accordingto example embodiments may provide the prediction result about the fatalsymptoms based on the pretrained machine learning model. If evaluationinformation about the prediction result is used as data to retrain themachine learning model, the machine learning model may perform a furtheraccurate prediction. Therefore, the prediction result generating methodaccording to example embodiments may further include operation S450 ofupdating, by the processor 220, or supporting the other apparatus toupdate the machine learning model based on evaluation information aboutthe prediction result. Here, individuation data (acquired from vitalsigns) not used for previous learning may be further considered and anerror found in the previous learning may be corrected. Therefore, theaccuracy of the machine learning model may be enhanced. As dataaccumulates, the performance of machine learning continues to improve.

Here, the evaluation information about the prediction result may beprovided from the external entity, such as the medical specialist, etc.

Such update refers to proceeding with learning again based on newlyprovided data and thus, may be substantially identical to theaforementioned operation S405. That is, the analysis model and theprediction model using the analysis modules 330 a and 330 b and theprediction module 340 are modified by considering the accuracy ofprediction based on the evaluation information about the predictionresult. In more detail, parameters, for example, the U, V, W, etc., usedfor the analysis model and the prediction model are modified.

The example embodiments may further quickly and conveniently predictfatal symptoms compared to a conventional method of predicting fatalsymptoms, such as cardiac arrest, sepsis, etc., generally depending onexperience or knowledge of medical specialists.

Advantages of the example embodiments described herein may significantlyreduce burden of medical specialists in busy medical clinical conditionsthat they need to make an accurate determination and prediction based ona large amount of diagnostic data a day. Using technology based onmachine learning, particularly, a deep neural network (DNN), the risk ofa patient related to fatal symptoms acquired by a doctor only afteryears of training may be analyzed and learned using a computingapparatus itself based on a large amount of learning data. Accordingly,assistance may be made to determine cases that a human medicalspecialist may overlook or cases in which it is difficult to predictfatal symptoms. For example, according to the example embodiments, onlypatients of which fatal symptoms are suspected to occur based on theautomatically generated prediction information, for example, by apredetermined point in time may be screened and medical staff may needto verify the screened patients. Accordingly, it is possible to improvethe accuracy and speed of prediction of fatal symptoms.

FIG. 5 illustrates an example of a method of performing a dataaugmentation using a modified generative adversarial network (GAN)according to an example embodiment.

The example embodiment relates to outperforming a class imbalance issuefound in the conventional machine learning, that is, to enhancing areliability and accuracy of a machine learning model since theconventional machine learning is performed mainly based on databelonging to a majority class due to a class imbalance.

To this end, in the example embodiment, the following data augmentationmay be performed. Referring to FIG. 5 , the modified GAN according tothe example embodiment may include a generator (indicated with ‘G’)configured to generate vital signs of fatal symptoms similar to realityand a discriminator (indicated with ‘D’) configured to discriminate truedata from generated data.

In detail, according to the paper regarding the conventional GAN,non-patent document 1: [Goodfellow, Ian J.; Pouget-Abadie, Jean; Mirza,Mehdi; Xu, Bing; Warde-Farley, David; Ozair, Sherjil; Courville, Aaron;Bengio, Yoshua (2014). “Generative Adversarial Networks”], a generatoris configured to generate data similar to true data to deceive adiscriminator, such that the discriminator may determine the similardata as the true data, and the discriminator is configured todiscriminate the true data from the generated similar data. Duringprogress of learning by the GAN, each of the generator and thediscriminator updates a network weight to achieve each correspondingpurpose. Accordingly, after sufficient learning, the generator maygenerate data similar to true data and a discrimination rate by thediscriminator may converge theoretically to 0.5.

Accordingly, since the generator sufficiently trained through theconventional GAN may generate data close to true data, theaforementioned class imbalance issue of data may be overcome by usingthe similar data generated by the generator of the conventional GAN aslearning data for training the machine learning model.

The conventional GAN focuses on learning simply depending whether theinput data is true data or fake data. Thus, if the input data may belongto various labels, the conventional GAN may not readily verify a labelcorresponding to the input data among the various labels.

To outperform the above limit, the modified GAN modified from theconventional GAN is used herein. Therefore, various kinds (i.e., kindsclassified by labels) of data may be generated and determined by furtherconsidering information of a label related to data using the modifiedGAN.

In detail, dissimilar to the conventional GAN, the generator of themodified GAN may include sub-generators (indicated with ‘G_label1’,‘G_label2’, and ‘G_label3’ in FIG. 5 ) each configured to generatesimilar data corresponding to a plurality of labels, respectively. Eachsub-generator generates similar data belonging to a label correspondingto the sub-generator.

Also, dissimilar to the conventional GAN, the discriminator of themodified GAN predicts and discriminates a label corresponding to data tobe determined by the discriminator instead of simply determining whetherthe data to be determined by the discriminator is true data or fakedata.

For example, such labels may include ‘cardiac arrest group’, ‘sepsisgroup’, ‘normal’, and the like. Therefore, the modified GAN according toexample embodiment may support data generated by the generator to becomesimilar data close to true data, and may support the discriminator toverify a label corresponding to the true data or the similar data inputto the discriminator, such that the generator may generate a specificlabel through the sub-generator.

That is, the modified GAN according to the example embodiment may betrained based on true data and a kind of a label corresponding to thetrue data and accordingly, may generate various kinds of similar dataclassified by labels to be close to true data. In particular, withrespect to a specific label having a relatively small quality of truedata, the modified GAN may replenish an insufficient quantity bygenerating similar data corresponding to the specific label and therebysolve a data imbalance issue about the specific label.

Hereinafter, a data classification method according to an exampleembodiment is described based on the aforementioned description. Thedata classification method includes operation S100′ of acquiring, by thecomputing apparatus 200 through the communicator 210, or supportinganother apparatus interacting with the computing apparatus 200 toacquire true data. For example, such true data may be a time seriessignal, however, without being limited thereto, any data to beclassified may be included in the true data.

The data classification method further includes operation S200′ oftraining, by the computing apparatus 200 through the processor 210, orallowing the other apparatus interacting through the communicator 210 totrain the generator and the discriminator of the modified GAN based onlabel information of a label corresponding to the acquired true data andthe true data. A difference between the modified GAN and theconventional GAN is described above and further description is omitted.

The data classification method further includes operation S300′ oftraining, by the computing apparatus 200 through the processor 220, orallowing the other apparatus to train the machine learning model bygenerating the similar data using the trained modified GAN and by using(i) the true data and the similar data or (ii) the similar data aslearning data of a predetermined machine learning model forclassification.

For example, a convolutional neural network (CNN), a recurrent neuralnetwork (RNN), and the like may be included in the machine learningmodel. However, it is provided as an example only, which may beunderstood by those skilled in the art.

Also, the data classification method further includes operation S400′of, in response to acquiring data to be classified by the communicator210 of the computing apparatus 200, generating, by the computingapparatus 200 through the processor 220, or supporting the otherapparatus to generate classification information of the data to beclassified by classifying the data to be classified based on the machinelearning model.

Further, the data classification method may further include operationS500′ of providing, by the computing apparatus 200 through the processor220, or supporting the other apparatus to provide the classificationinformation to an external entity. Here, the external entity may includea user of the computing apparatus, an administrator, and the like. Inaddition thereto, it should be understood that any entity having a rightto acquire the classification information may be included.

Also, the data classification method may further include operation S600′of updating, by the computing apparatus 200, or supporting the otherapparatus to update the machine learning model based on evaluationinformation about an accuracy of the classification information.

The aforementioned data classification method according to the exampleembodiment may provide classification information about data to beclassified based on a predetermined machine learning model. Therefore,in the case of using evaluation information about an accuracy ofclassification information as retraining data, a further accurateprediction may be performed. Accordingly, the data classification methodmay further include operation S600′ of updating, by the computingapparatus 200 through the processor 220, or supporting the otherapparatus to update at least one of the machine learning model and themodified GAN based on evaluation information about the classificationinformation.

For example, operation S600′ may include an operation of directlyupdating, by the computing apparatus 200, or supporting the otherapparatus to update the machine learning model based on the evaluationinformation about the classification information or an operation ofindirectly updating or supporting the other apparatus to update themachine learning model using data generated by the generator of themodified GAN by training the modified GAN based on the evaluationinformation about the prediction information.

Here, learning data not used for previous learning may be furtherconsidered and an error found in the previous learning may be corrected.Therefore, the accuracy of the modified GAN may be enhanced. As dataaccumulates, the classification performance of the machine learningmodel continues to improve. Also, according to example embodiments,errors in true data being provided may be significantly reduced and aclass imbalance of learning data used to train the machine learningmodel may be solved. Therefore, the reliability of the trained machinelearning model may be improved.

According to the example embodiments, although a class imbalance oflearning data is high in machine learning, it is possible to enhance theaccuracy of a classification model by existing machine learning througha data augmentation of generating learning data of a minority classsimilar to true data.

Based on the description of the example embodiments, those skilled inthe art may clearly understand that the disclosure may be implementedusing a combination of software and hardware or hardware alone. Targetsof technical solutions of the disclosure or portions contributing to thearts may be configured in a form of program instructions performed byvarious computer components and stored in non-transitorycomputer-readable recording media. The media may include, alone or incombination with the program instructions, data files, data structures,and the like. The program instructions recorded in the media may bespecially designed and configured for the example embodiments, or may beknown to those skilled in the art of computer software. Examples of themedia may include magnetic media such as hard disks, floppy disks, andmagnetic tapes; optical media such as CD-ROM discs and DVDs;magneto-optical media such as floptical disks; and hardware devices thatare specially configured to store and perform program instructions, suchas ROM, RAM, flash memory, and the like. Examples of programinstructions may include a machine code, such as produced by a compilerand higher language code that may be executed by a computer using aninterpreter.

The hardware devices may be configured to operate as at least onesoftware module to perform processing of the example embodiments, orvice versa. The hardware devices may include a processor, such as a CPUor a GPU, configured to combine with a memory, such as ROM/RAM, forstoring program instructions and to execute the instructions stored inthe memory, and may include a communicator configured to transmit andreceive signals to and from an external apparatus. Further, the hardwaredevices may include a keyboard, a mouse, and other external inputdevices for receiving instructions produced by developers.

According to some example embodiments, compared to an existing method ofdetermining the risk of a patient depending on experience or knowledgeof medical specialists, it is possible to further quickly and easilypredict fatal symptoms, such as cardiac arrest, sepsis, etc.

Also, according to some example embodiments, compared to an existingmethod of assigning a rule-based score, it is possible to furtherdecrease false negatives and false positives in predicting fatalsymptoms.

Also, according to some example embodiments, it is possible to innovatea workflow in a medical field by enabling a continuous prediction intime-series clinical processes.

Also, according to some example embodiments, a prediction performancemay be continuously improved by using a method of this disclosure.

Also, according to some example embodiments, it is possible tooutperform a class imbalance of learning data that is an issue in aclassification model by existing machine learning, that is, in traininga machine learning model for classification, through a data augmentationof learning data.

For example, a data classification method using data augmentationaccording to example embodiments may apply to many medical determinationsituations and thus, may enhance a low accuracy of a classificationmodel caused by minority data related to cardiac arrest, sepsis, etc.,since most subjects are subjects not showing corresponding symptoms.

While this disclosure is described with reference to specific matterssuch as components, some example embodiments, and drawings, they aremerely provided to help general understanding of the disclosure and thisdisclosure is not limited to the example embodiments. It will beapparent to those skilled in the art that various alternations andmodifications in forms and details may be made from the exampleembodiments.

Therefore, the scope of this disclosure is not defined by the exampleembodiments, but by the claims and their equivalents, and all variationswithin the scope of the claims and their equivalents are to be construedas being included in the disclosure.

The equally or equivalent modifications may include, for example, alogically equivalent method that may achieve the same result as oneacquired by implementing the method according to this disclosure.

What is claimed is:
 1. A method for a computing apparatus operatingbased on a machine learning model, the method comprising: acquiringfirst vital signs of a subject during a first time duration; predictingnormal vital information related to a normal state of the subject basedon the first vital signs; converting the acquired first vital signs intosecond vital signs that are individualized to fit a characteristic ofthe subject based on the normal vital information; generatingindividuation data based on the individualized second vital signs;generating analysis information about a prediction of fatal symptomsfrom the individuation data based on the machine learning model; andgenerating the prediction result which is determined by a result ofpredicting occurrence of the fatal symptoms during a predeterminedduration based on the analysis information.
 2. The method of claim 1,wherein the normal vital information includes a characteristic value forthe normal state of the subject predicted based on the first vitalsigns.
 3. The method of claim 2, wherein the second vital signs areconverted based on a difference between the characteristic value and thefirst vital signs.
 4. The method of claim 3, wherein the characteristicvalue is calculated based on an average value of at least one firstvital signal corresponding to a second time duration among the firstvital signs, and wherein the second time duration is a partial timeduration predicted that the subject is in the normal state in the firsttime duration.
 5. The method of claim 1, further comprising: updatingthe machine learning model based on evaluation information about theprediction result.
 6. The method of claim 1, wherein the first vitalsigns are acquired from an electronic medical record (EMR) of thesubject.
 7. The method of claim 1, wherein the individuation data isgenerated by; generating individuation data for the subject bycalculating a standard score (z-score) for the second vital signs withreference to an average and variance of vital signs of other subjects.8. The method of claim 1, wherein the machine learning model comprisesan analysis model comprising a recurrent neural network model as ananalysis model, the recurrent neural network model followss_(t)=f(Ux_(t)+Ws_(t-1)) and y=g(Vs_(t)), where the x_(t) denotes theindividuation data that is an input vector at a point in time t or avalue processed from the individuation data, the s_(t) denotes a hiddenstate corresponding to a memory of the recurrent neural network model atthe point in time t, the s_(t-1) denotes the hidden state at the pointin time t−1, the U, V, and W denote neural network parameters sharedequally across all the points in times of the recurrent neural networkmodel, the f denotes a predetermined first activation function that isselected to calculate the hidden state, the y denotes an output layerthat is a latent feature according to the recurrent neural network modelat the point in time t as the analysis information, and the g denotes apredetermined second activation function that is selected to calculatethe output layer, and the machine learning model further comprises aprediction model comprising at least one fully connected layer forcalculating an occurrence probability of the fatal symptoms from theoutput layer.
 9. The method of claim 1, wherein the fatal symptomscomprise an occurrence of cardiac arrest or sepsis.
 10. The method ofclaim 1, further comprising: providing the generated prediction resultto an external entity.
 11. A non-transitory computer-readable storagemedium storing a program instructions that is executable by a computerto perform the method of claim
 1. 12. A computing apparatus operatingbased on a machine learning model, the computing apparatus comprising: acommunicator; and a processor configured to communicate through thecommunicator wherein the processor is configured to: acquire first vitalsigns of a subject during a first time duration, predict normal vitalinformation related to a normal state of the subject based on the firstvital signs convert the acquired first vital signs into second vitalsigns that are individualized to fit a characteristic of the subjectbased on the normal vital information, generate individuation data basedon the individualized second vital signs, generate analysis informationabout a prediction of the fatal symptoms from the individuation databased on the machine learning model, and generate the prediction resultdetermined by a result of predicting occurrence of the fatal symptomsduring a duration based on the analysis information.
 13. The apparatusof claim 12, wherein the processor is configured to update the machinelearning model based on evaluation information about the predictionresult.
 14. The apparatus of claim 12, wherein the machine learningmodel comprises: an analysis model comprising a recurrent neural networkmodel; and a prediction model, wherein the recurrent neural networkmodel follows s_(t)=f(Ux_(t)+Ws_(t-1)) and y=g(Vs_(t)), where the x_(t)denotes the individuation data that is an input vector at the point intime t or a value processed from the individuation data, the s_(t)denotes a hidden state corresponding to a memory of the recurrent neuralnetwork model at the point in time t, the s_(t-1) denotes the hiddenstate at the point in time t−1, the U, V, and W denote neural networkparameters shared equally across all the points in times of therecurrent neural network model, the f denotes a predetermined firstactivation function that is selected to calculate the hidden state, they denotes an output layer that is a latent feature according to therecurrent neural network model at the point in time t as the analysisinformation, and the g denotes a predetermined second activationfunction that is selected to calculate the output layer, wherein theprediction model comprising at least one fully connected layer forcalculating an occurrence probability of the fatal symptoms from theoutput layer.
 15. The apparatus of claim 12, wherein the fatal symptomscomprise an occurrence of cardiac arrest or sepsis.